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I.  IMTRODUCTION 

As  a  part  of  the  Weapons  Director  Workload  Analysis  Project  (Ref.  l),  an 
application  of  the  critical  incident  technique  (Ref.  2)  \ra.s  made  to  obtain 
data  which  could  be  analyzed  in  identifying  problem  areas  of  the  Weapons 
Director  (WD)  function.  Identification  of  these  areas  provided  a  basis 
for  recommendations  in  improving  the  effectiveness  of  the  Weapons  Director 
operations.  The  specific  recommendations  generated  as  a  result  of  this 
study  are  not  reported  in  this  document,  but  have  been  published  in  the 
final  report  (Ref.  3)  of  the  project  group. 

In  approaching  the  task  set  forth,  it  -svas  decided  to  examine  three  aspects 
of  the  WD  function.  These  aspects  were: 

1.  The  perfojrmance  requirements  of  the  position. 

2.  The  WD's  interactions  with  the  Senior  Weapons  Director 
( SWD)  and  the  Intercept  Directors  ( INDs ) . 

3.  The  console  equipment,  communications  equipment,  computer 
programs  and  standard  operating  procedures  with  which  the 
WD  is  concerned. 

More  specifically,  the  study  sought  to  accomplish  the  following  objectives 

1.  To  obtain  incidents  of  effective  and  ineffective  behaviors 
in  the  performance  of  the  WD  position. 

2.  To  obtain  incidents  of  effective  and  ineffective  behaviors 
of  the  SWD  and  the  INDs  in  their  interactions  with  the  WD. 

3.  To  obtain  incidents  concerning  situations  in  which  the  WD 
was  hindered  in  the  effective  performance  of  his  job  by 
limitations  or  constraints  imposed  by  the  console  equipment, 
communications  equipment,  computer  programs  and  standard 
operating  procedures  related  to  his  position. 

4.  To  systematically  collect  and  analyze  these  incidents  in 
identifying  problem  areas  and  to  provide  a  basis  for 
recommendations  in  improving  the  function. 


II.  PROCEDURE 

A.  Data  Source 


Critical  incidents  were  obtained  from  respondents  (SWDs,  WDs,  and 
INDs)  at  six  SAGE  Air  Defense  Sector  (ADS)  Direction  Centers: 

Duluth  ADS,  Cliicago  ADS,  Seattle  ADS,  Portland  ADS,  New  York  ADS, 
and  Washington  ADS.  These  sites  were  selected  to  obtain  an  incident 
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sample  ^ich  would  represent  the  different  sites  operating  in  various 
environments  and  geographical  locations. 

B.  Preparation  of  the  Questlonnni  re  Form 

A  review  was  made  of  some  studies  in  which  the  critical  incident 
technique  was  used.  These  studies  furnished  ideas  on  the  construction 
of  forms  for  the  collection  of  incidents.  For  the  present  study,  a 
form  developed  by  Wagner  (Ref.  4^  was  adapted  for  use.  The  forms 
contain  one  section  for  the  reporting  of  the  incident  and  a  second 
section  for  questions  regarding  the  respondent,  the  incident,  and  the 
person  involved.  The  questions  used  in  the  second  section  were  con¬ 
structed  so  the  anonymity  of  the  respondent  and  the  person(s)  involved 
could  be  maintained.  The  items  contained  in  this  section  permit  the 
analysis  of  the  relationship  between  the  reported  behaviors  and  certain 
characteristics  of  the  respondent,  the  person  involved,  etc. 

Six  questlonhaire  forms  were  constructed  for  use  in  the  present  study. 
Five  of  the  forms  were  designed  in  the  general  manner  described  above, 
while  a  sixth  (the  WD  Operational  Procedures  and  Equipment  Questionnaire) 
was  constructed  without  the  sample  control  data  section. 

An  example  of  the  critical  Incident  form  is  contained  in  Appendix  A. 

Table  I,  shown  on  the  next  page,  presents  a  list  of  the  forms  used, 
their  purposes  and  the  appropriate  respondent  for  each  of  the  forms. 

C.  Preliminary  Administration  and  Checkout 

Prior  to  the  administration  of  the  questionnaires  at  the  sites  mentioned 
earlier,  an  initial  set  of  the  forms  was  administered  to  Weapons 
personnel  at  the  Los  Angeles  Air  Defense  Sector.  As  a  result  of  this 
preliminary  administration,  several  suggestions  were  obtained  regarding 
the  Interview  procedure  and  the  questionnaire  forms.  On  the  basis  of 
these  suggestions,  several  changes  were  incorporated  in  the  final 
questionnaire  forms  and  interview  procedure. 

D.  Data  Collection  Method 

1.  Administrative  Arrangements 

The  Field  Operations  Department  (FOD)  representative  in  the  project 
group  established  liaison  with  the  sites  Involved.  SDC  field 
site  personnel  were  furnished  with  the  appropriate  incident  forms, 
a  guide  to  the  administration  of  these  forms  and  a  suggested  brief¬ 
ing  outline.  See  Appendix  C  for  the  guide  and  briefing  outlines. 

A  supporting  memorandum  from  FOD  outlining  the  objectives  and  the 
purpose  of  the  study  accompanied  the  package  of  fomns. 
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FORMS  USED  IN  THE  COLLECTION  OF  CRITICAL  INCIDENTS 


Form  Name 

Purpose 

Respondent 

(Person  to  whom  questJoii’ 
naire  was  administoreO.) 

T  , 

Critical  Incident  Form 
WD-SWD-(E) 

(I) 

To  collect  incidents 
of  effective  (E)  and 
ineffective  (I)  WD 
behavior. 

Senior  Weapons 

2. 

Critical  Incident  Form 
WD-IND-(E) 

(I) 

To  collect  incidents 
of  effective  and  in¬ 
effective  WD  behavior. 

Intercept  Dirsctoi' 

3. 

Critical  Incident  Form 
',/D-(E) 

(I) 

To  collect  incidents 
of  effective  and  in¬ 
effective  WD  behavior. 

Weapons  Director 

1^. 

Critical  Incident  Form 
Iin)-WD-(E) 

To  collect  Incidents 
of  IND  behaviors  which 

Weapons  Director 

(I) 

helped  or  hindered  t}ie 
WD  in  performing  hi.: 
job  effectively. 

5. 

Critical  Incident  Form 
SWD-WD-(E) 

(I) 

To  collect  incidents 
of  SWD  behaviors  which 
helped  or  hindered  the 
WD  in  performing  his 
.job  effectively. 

Weapons  Director 

6.  Weapons  Director 

Operational  Procedures 
and  Equipment  Question¬ 
naire 


To  collect  incidents 
in  which  the  WD  'rfas 
hindered  in  the  per¬ 
formance  of  his  job  due 
to  constraints  or 
limitations  of  equipment, 
computer  program,  or 
standard  operating  pro¬ 
cedures  . 


Weapons  Director 
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D.  Data  Collection  Method  (Cont'd) 

A  letter  from  the  Air  Defense  Command  Computer  Programing  and 
System  Training  Office  was  issued  to  the  military  commander  at 
each  of  the  selected  sites.  The  letter  was  intended  to  inform 
the  military  personnel  of  the  purpose  of  the  study  and  to  elicit 
their  cooperation  in  the  accomplishment  of  the  study’s  objectives. 

2.  Interview  Procedure 

A  group  briefing  followed  by  individual  interviews  was  suggested 
to  the  field  site  interviewers  as  a  procedure  in  gathering  the 
data.  This  procedure  was  found  to  be  more  effective  in  incident 
production  from  the  preliminary  tryout  at  LAADS.  If  a  group 
briefing  did  not  occur,  the  interviewers  were  instructed  to  give 
the  briefing  individually  prior  to  the  administration  of  the 
questionnaires . 

Briefings  were  given  to  the  respondents  for  the  following  purposes: 

a.  To  explain  the  nature  and  purpose  of  the  study. 

b.  To  give  instructions  for  filling  out  the 
questionnaires . 

c.  To  give  exan5>les  of  v/ell  vrritten  and  poorly  written 
incident  reports  to  aid  the  respondents  in  identifying 
and  writing  out  the  cx’iticnl  incidents. 

By  directing  the  respondents'  attention  to  specific  outcomes  of 
effective  or  ineffective  behaviors,  it  becomes  possible  to  isolate 
and  describe  the  behavior(s)  responsible  for  that  outcome.  The 
"meaning"  of  critical  incidents  as  vrell  as  the  requirements  for 
writing  the  reports  can  thus  be  made  clear. 

III.  ANALYSIS  OF  THE  DATA 

A.  Treatment  of  the  Data  Obtained  from  the  BeJinvioral  iieports 

The  treatment  of  the  data  obtained  from  the  questionnaire  forms  on 
which  behavioral  reports  were  made  (i.e.,  Forms  1  through  5  in  Table  I) 
is  described  in  this  section.  The  data  obtained  from  the  WD  Operational 
Procedure  and  Equipment  Questionnaire  v/ere  treated  separately  and  are 
discussed  in  the  section  to  follow. 

1.  Transcription  and  Coding  of  the  Data 

Eswih  incident  from  the  questionnaire  form  was  transcribed  and 
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typed  separately  on  a  5  X  8  McBee  Keysort  card.  After  all  Incidents 
had  been  transcribed,  the  8aog;)le  control  data  (information  from 
the  lower  section  of  the  questionnaire)  were  coded  on  the  cards 
and  the  appropriate  columns  were  punched  in  accordance  with  a 
coding  scheme.  The  cards  were  nia&bered  consecutively  In  the  same 
manner  as  the  questionnaire  forms  from  which  the  Incidents  were 
transcribed. 

2.  Isolation  of  Critical  Behaviors  and  a  Check  for  Usability 

Each  of  the  Incidents  on  the  McBee  CEurds  was  read  and  the  critical 
behavior  underlined.  In  cases  \diere  several  different  behaviors 
were  involved  in  an  incident,  each  of  the  separate  behaviors  was 
underlined  and  noted.  Ihls  allowed  an  Incident  card  containing 
different  behaviors  to  be  classified  under  more  than  one  category. 

During  the  process  mentioned  above,  a  check  was  made  to  eliminate 
those  Incident  reports  which  were  Judged  not  usable.  !niese  Included 
reports  that:  l)  were  unclear  in  meaning,  2)  listed  traits  rather 
thiui  actucLL  behaviors  observed  In  specific  situations,  3)  contained 
Incidents  In  idiich  equipment  or  procedural  deficiencies  were  sub¬ 
stituted  for  behaviors. 

3.  Formulation  of  Crude  Cate«orles  of  Behavior 

The  usable  Incident  cards  with  the  behaviors  underlined  were  reread 
for  the  purpose  of  Identifying  those  incidents  In  which  slmlleur 
behaviors  were  Involved,  ^ese  cards  were  grouped  together  suid  a 
description  of  the  behavioral  similcurltles  was  written  on  a  sheet 
of  paper.  TOie  appropriate  card  numbers  were  written  beside  the 
description.  Cards  with  more  than  one  behavior  underlined  were 
then  removed  so  they  could  be  Included  In  other  groupings,  ^ese 
groupings  constituted  the  Initial  formulation  of  crude  categories. 

The  process  was  repeated  until  all  cards  were  categorized.  After 
the  InltlEa  sort  had  been  made  in  the  msmner  described,  the  entire 
procedure  was  repeated  for  the  following  purposes:  l)  to  Insure 
that  the  categories  were  as  mutually  exclusive  as  possible,  2)  to 
make  Emy  necessary  corrections  or  revisions  of  the  category  descrip¬ 
tions,  and  3)  to  check  on  any  errors  (e.g. ,  placing  cards  in  the 
wrong  category,  etc.)  In  the  initial  sort. 

4.  Beflnlng  and  Forming  the  Final  Categories 

Fbur  members  (3  civili««i,  1  military)  of  the  project  group  sat  In 
conference  to  review  the  category  descriptions  and  the  placement 
of  Incidents  in  eeich  category.  Essentlsklly,  the  procedure  consti¬ 
tuted  a  third  sorting  of  the  behaviors  Into  categories.  The  Incidents 
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were  read  and  a  consensus  was  obtained  in  regard  to  the  placement 
of  any  card  in  a  given  category.  The  category  descriptions  were 
checked  for  use  of  correct  terminology  and  for  appropriateness. 
Categories  were  combined  and  new  categories  were  made  when  necessary. 

B.  Treatment  of  the  Data  from  the  WD  Operationeil  Procedures  and 
Equipment  Questionnaire 

Olie  incidents  obtained  from  this  questionnaire  were  typed  on  5  X  8 
McBee  cards  and  coded  with  respect  to  the  relevant  problem  Eurea 
(e.g. ^  console  equipment ^  communication  equipment^  computer  program 
and  standard  operating  procedures).  The  cards  were  then  submitted 
to  the  project  group  representatives  of  the  Requirements  and  Design 
Branch  (RDB)  for  analysis. 

Each  of  the  Incidents  was  resui  and  a  list  (Ref.  5)  of  the  different 
problems  and  the  corresponding  suggestions  for  Improvement  was  com¬ 
piled.  The  list  of  problems  and  the  related  suggestions  were  divided 
into  three  categories.  The  first  category  included  those  problems 
and  suggestions  on  which  action  had  been  taken  through  appropriate 
SPCs  (SAGE  Program  Changes)  or  ECPs  (Engineering  Change  Proposals). 

The  second  category  contained  those  problems  which  were  not  covered 
by  current  changes  and  for  which  corrective  actions  would  be  recommended. 
The  final  category  Included  the  problem  statements  and  suggestions  that 
were  rejected.  Appropriate  comments  Indicating  the  reason  for  rejection 
were  attached  to  each  of  the  statements. 

IV.  RESULTS 

A.  Categories  of  Job  Performance  Behaviors 

From  a  total  of  158  usable  incident  reports,  I88  behaviors  were 
obtained  and  categorized.  For  the  Job  performance  behaviors  of  the 
Weapons  Director  position,  115  behaviors  were  placed  in  22  categories. 
For  the  Senior  Weapons  Director  position  35  behaviors  were  obtained 
and  9  behavior  categories  were  formed.  Thirty-eight  behaviors  were 
obtained  concerning  the  IMD  position  and  12  categories  of  behaviors 
were  formed.  The  fineU.  set  of  categories  of  performance  behaviors 
obtained  in  the  study  is  presented  in  the  pages  to  follow. 
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Ineffective  Job  Performance  Total 

of  Weapons  Directors  Freouenc 


,  Made  poor  assignment  of  Inter¬ 
ceptors  to  IMPS  for  control. 

He  assigned  Interceptors  to  an  10 

IND  without  regard  to  area, 
causing  the  IND  to  contlnxially 
switch  his  display  to  different 
sections  of  the  sector;  restrict¬ 
ing  the  IND*  8  use  of  the  X8  esqianded 
mode;  and  necessitating  the  IND  to 
switch  to  different  radio  sites  to 
maintain  effective  voice  control. 


Frequency  by  Reporting  Source 
WD 


2.  Meuie  inappropriate  commitments  of 
interceptors  to  targets. 


He  mad.e  inappropriate  interceptor- 
target  commitments  by  neglecting 
to  utilize  the  Veai>ons  Assignment 
displays  and  made  pairing  idiich 
immediately  resulted  in  in^osslble 
intercept  conditions  according  to 
program  calculations;  he  exercised 
X>oor  Judgment,  selecting  aircraft 
in  a  less  eidvantageous  position  to 
effect  €ui  intercept  prior  to  the 
bomb  release  line. 

3.  Failed  to  distribute  assignments  and 
maintain  even  loads  on  the  INDs. 

He  distributed  interceptors  for 
control  and/or  intercept  missions 
unevenly  among  his  INDs,  causing  cm 
overload  on  some  INDs,  vhlle  other 
INDi?  were  available  with  little  or 
no  work  to  do. 

4.  Did  not  pass  on  important  information 
required  by  others  in  the  performance 
of  their  functions. 

He  failed  to  provide  others  with 
information  about  sui  impending 
situation,  changes  in  prior  Instruc¬ 
tions  or  plans,  or  in^rtant  factors 
In  the  current  situation  ^en  being 
relieved  from  duty. 
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Ineffective  Job  Performance  Total  Frequency  by  Reporting  Source 

of  Weapons  Directors _ Frequency _ SVTO _ WD  IND 

5.  Passed  incomplete  or  Incorrect 
infoimatlon  to  others  vlthout 
making  subsequent  effort  to 
complete  or  correct  the  in¬ 
formation. 

He  passed  incomplete  or  incorrect  3  111 

Information  because  he  lacked 
sufficient  knowledge  of  the  require¬ 
ments;  did  not  know  to  whom  he  should 
direct  the  information;  and  did  not 
understand  the  origlncJ.  information 
given  to  him. 

6.  Interfered  with  the  functions  of 

the  IND  by  excessive  and  Inappropriate 
use  of  the  tactical  lines. 

He  would  call  to  check  on  items  which  2  2 

were  already  acconq>llshed  and  could 

be  checked  without  making  a  call;  to 

determine  whether  instructions  h^ 

been  carried  out  without  allowing 

time  for  their  accomplishment;  and 

to  inquire  about  information  which 

could  be  more  readily  obtained  from 

other  sources  (e.g.,  SIDs  and  DIDs). 

He  would  make  "unnecessary"  calls 
attributable  to  his  own  lack  of 
knowledge. 

7.  Operated  switches  and/or  equipment 
incorrectly  or  failed  to  employ  them 
properly  while  operating  at  his 
console  position. 

He  did  not  take  the  proper  console  5  1  13 

switch  actions  because  he  lacked 
knowledge  of  standard  operating 
procedures  and/or  regulations  re¬ 
lated  to  the  operations^,  functions 
of  the  console. 
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Inei'i'ecoive  Job  Performance 
of  Weapons  Directors 


Total  f'requency  by  Reportinc  Source 
Frequency  SWD  \!'D  Xi'fD 


8 .  Lost  control  of  a  situation  for 
vhich  he  vas  responsible. 

He  alloved  a  situation  to  get  out  10 

of  control  due  to  the  exercise  of 
poor  judgment,  lack  of  knowledge 
about  the  situation,  and  inadequate 
use  or  lack  of  effective  plans  and 
actions  in  coping  with  the  situation. 


9' 


Took  action  without  informing 
personnel  who  would  be  concerned 
with  or  affected  by  the  action. 


He  failed  to  inform  relevant 
personnel  before  acting,  causing 
a  compounding  of  the  original 
problem  and/or  creating  new  problems. 


3 


10. 


Became  preoccupied  in  performing  a 
task  which  was  the  responsibility  of 
another  person  and  neglected  to  dis- 
charge  his  own  responsibilities. 


He  assumed  responsibility  for  per¬ 
forming  a  task  ordinarily  done  by 
another  person  in  an  attempt  to  do 
it  better  or  to  improve  a  situation 
with  which  the  performance  of  the 
task  was  concerned,  but  in  doing  so 
he  neglected  to  adequately  fulfill 
hie  own  responsibilities. 


Totals  for  Ineffective  Job  5**^ 

Performance  of  WDs 
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Effective  Jo'o  Performance  of  Total  Frequency  by  Heportinfj;  So  iree 

_ 'Weapons  Directors  Prequcncy  SWD  ,.’D _ IIIJ 

11 .  Aspinted  TEDg  In  handling  emergency 
cituiit j.oriG  by  taking  prompt  and 
effective  actions. 

V/hen  an  emergency  situation  developed  71  o 

he  acted  without  delay,  relieving 
the  'OiD  of  the  control  of  other  air¬ 
craft;  coordinating  with  the  re¬ 
cover,/  arer,  and  ground/emergency 
faci]  !ti.caj  providing  a  steady  flow' 
of  pertinent  information  to  the  lifl); 
and  maintaining  effective  monitor- 
in{5  of  the  situation. 

12.  Directed  II=IDs  to  make  their  oim 
coinrnltments  (pairings)  when  fa~e 
anticipated  or  recognized  a  satu¬ 
rated  display  condition  developing 
at  his  console  position. 

He  directed  the  INDs  to  make  their  5  5 

own  pairings  to  prevent  the  delay 

of  timely  and  effective  commitments 

which  might  result  from  the  saturated 

display  condition  at  his  console. 

13.  Effectively  monitored  the  control 
of  aircraft  by  the  INDs. 

He  effectively  monitored  the  con-  10  10 

trol  of  aircraft  being  conducted  by 
his  INDs  by  recognizing  situations 
and/or  conditions  which  would  pro¬ 
vide  difficulties  and  by  taking 

appropriate  actions  or  passing  perti-  _ 

nent  information  which  would  help 
alleviate  these  difficulties. 

lU.  Made  effective  assignment  of  inter¬ 
ceptors  to  INDs  for  control. 

He  assigned  interceptors  to  INDs  21  1 

for  control  in  areas  of  close  prox¬ 
imity,  enabling  the  INDs  to  monitor 
and  conduct  Intercepts  with  a  mini¬ 
mum  amount  of  switch  changing  actions 
(e.g.,  e^and  modes,  radio  sites, 
etc . ) . 
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Effective  Job  Performance  of 
Weapons  Directors _ 


Total 

Frequency 


Frequency  by  Reporting  Source 
SWD _ ra  IW) 


15.  Distributed  assignments  and  main¬ 
tained  even  loads  on  the  IWDs. 

He  maintained  efforts  to  distribute  6 

assignments  and  keep  even  loads  on 
all  IKDs  under  his  supervision. 

16.  Made  ej.npronriate  comifiltments  of 
Interceptors  to  targets. 

He  made  appropriate  interceptor-  4 

target  commitments  by  utilizing 
the  proper  information  displays 
and  substantially  reduced  recommit 
actions;  he  exercised  good  judgment 
and  selected  aircraft  in  the  most 
advantageous  position  to  effect  an 
intercept  before  the  bomb  release 
line. 

17.  Formulated  and  executed  an  effective 
vork  plan  for  his  weapons  team. 

He  formulated  a  work  plan  for  his  9 
weapons  team  and  carried  it  out 
successfully  by  providing  appropri¬ 
ate  instructions  to  the  personnel 
concerned;  by  effective  scheduling; 
by  effective  monitoring  of  the 
operation  of  the  plan;  and  by  co¬ 
ordinating  with  the  personnel  con¬ 
cerned. 

18.  Passed  all  information  required  by 
others  in  the  performance  of  their 
functions . 

He  made  sure  that  he  provided  others  3 
with  all  information  regarding  an 
impending  situation;  regarding 
changes  in  prior  Instructions  or 
plans;  or  regarding  the  operation 
of  a  plan  or  a  schedule. 


1 


126 


1  2 
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Effectlve  Job  Performance  of 
_ Weapons  Directors 


Total 

Frequency 


Pregiuency  by  Reporting  Source 
8WD _ TO _ IMP 


19.  ^slsted  another  Weapons  Director 
(TO)  by  effectively  monitoring  the 
"gross'*  air  picture. 

By  not  confining  his  monitoring  10 

activity  solely  to  his  own  area  of 
responsibility,  he  was  In  a  posi¬ 
tion  to  aid  another  Weapons  Director 
by  sharing  the  workload;  by  com¬ 
mitting  more  euivantageously  posi¬ 
tioned  aircraft;  or  by  supplying 
Interceptor  aircraft  under  his  con¬ 
trol  to  another  WD  for  commitment. 

20.  Assumed  responsibility  to  perform  a 
task  ordinarily  done  by  someone  else 
and  was  successful  In  doing  so. 

He  acted  quickly  to  perform  a  task  2 
ordinarily  done  by  someone  else 
\dien  he  recognized  that  a  situation 
would  get  out  of  control  without 
immediate  action. 

21.  Acted  without  delay  In  correcting 
a  potentially  dangerous  condition. 

U)pon  recognition  of  a  condition  or  2 
situation  that  was  potentially 
dangerous,  he  quickly  took  the  re¬ 
quired  actlon(8)  to  prevent  or 
remove  the  potential  danger. 

22.  Took  steps  to  Insure  effective 
training  for  an  IND. 

He  insured  effective  training  for  1 
an  IHD  by  designating  an  appropri¬ 
ate  instructor;  by  selecting  assign¬ 
ments  appropriate  to  his  e]q)erlence 
level;  and  by  monitoring  the  IND's 
progress  effectively.  ___ 


2  8 


2 


2 


1 


Totems  for  Effective , Job 
Performance  of  WDs 


61 


7 


17 


37 
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Ineffective  Job  Performance  of  Intercept 

Directors  as  Related  to  the  WD  Position  Frequency 


1.  Did  not  follov  Instructions  or 
Ignored  advice  offered  to  him  In 
the  performance  of  his  Job. 

He  failed  to  follov  Instructions  or  4 

ignored  competent  advice  offered  to 
him  by  the  WD  which  resulted  in  an 
Inculequate  performance  of  the  task 
assigned  and  the  creation  of  addi¬ 
tional  problems. 

2.  Did  not  pass  on  Important  Information 
required  by  the  WD  In  the  performance 
of  his  .lob. 

He  failed  to  provide  the  WD  with  3 

Information  regarding  the  outcome 
of  an  Intercept  assignment;  regarding 
situation  that  would  affect  the 
operational  plan;  or  regarding  un¬ 
committed  aircraft  in  the  area. 

3.  Operated  switches  and/or  equipment 
Incorrectly  or  failed  to  use  them 
properly  ^lle  operating  at  his 
console  position. 

He  took  inqiroper  console  switch  actions  k 

or  used  the  wrong  equipment  because  he 
lacked  knowledge  of  standard  operating 
procedures  and/or  regulations  related 
to  the  operation  of  the  console  because 
he  lacked  knowledge  of  the  capabilities 
and  limitations  of  his  equipment. 

4.  Passed  on  Incomplete  or  Incorrect 
Information  without  making  subsequent 
effort  to  complete  or  correct  the 
Information. 

He  passed  on  Incomplete  or  Incorrect  1 

Information  because  he  lacked  sufficient 
knowledge  of  the  requirements;  did  not 
know  to  'vdiom  he  should  direct  the  informa¬ 
tion  and  did  not  understand  the  original 
Information  given  to  him. 


15  January  1963 


-l6- 


TM-956 


Ineffective  Job  Performance  of  Intercept 
Directors  as  Related  to  the  WD  Position 

Frecniency 

5.  Interfered  with  the  functions  of  the 
Weapons  Director  by  excessive  and 
inappropriate  use  of  the  tactical 
line. 

Because  he  lacked  knowledge  of 
standard  operating  procedures, 
regulations,  etc.,  related  to  the 
performance  of  his  task,  he  would 
csLll  at  inopportune  times  to  request 
information  which  he  should  have 
known. 

5 

6.  Did  not  position  his  aircraft 

properly  during  an  intercept  mission. 

He  failed  to  exercise  adequate  con¬ 
trol  in  positioning  his  aircraft  for 
an  intercept  because  of  poor  Judgment 
or  because  he  did  not  use  the  proper 
display  Information  necessary  to 
effect  a  good  position. 

5 

Total  for  Ineffective  Job 

Performance  of  INDs 

22 

Effective  Job  Performance  of  Intercept 
Directors  as  Related  to  the  WD  Position 

Frequency 

7-  Displayed  initiative  by  maneuverlnK 
aircraft  under  his  control  into 
advantageous  positions  for  ccanmitment 

When  he  recognized  a  flight  of  4 

threatening  aircraft  or  targets  of 
opportunity,  he  acted  on  his  own 
initiative  to  maneuver  his  inter¬ 
ceptors  into  favorable  positions 
for  effective  commitment  against 
the  targets. 
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Effective  Jot  Performance  of  Intercept 
Directors  as  Related  to  the  WD  Position 


8. 


Assisted  the  WD  by  making  ovn  commit¬ 


ments  (pairings)  on  target  aircraft. 


He  recognized  that  the  WD  vas  being 
hampered  by  a  saturated  display  conr 
dltlon  and  made  his  ovn  pairings  to 
prevent  the  delay  of  timely  and 
effective  commitments. 


9.  Demonstrated  resourcefulness  by 

conrpletlng  Intercent  although  handi¬ 
capped  by  computer  malfunction  or 
program  limitations. 

He  demonstrated  resourcefulness  by 
using  "manual  vectoring"  technlqiies 
to  complete  an  Intercept  when  he  vas 
hindered  by  computer  failure  or  pro¬ 
gram  limitations. 

10.  Demonstrated  alertness  by  recognizing 
a  condition  or  situation  vhich  could 
become  dangerous  and  informing  the 
responsible  WD  for  edition. 

He  demonstrated  alertness  by  detect¬ 
ing  a  potentially  dsuigerous  condition 
or  situation  and  by  acting  vlthout 
delay  to  inform  the  responsible  WD 
for  action. 


11.  Acted  effectively  in  handling  an 
emergency  situation. 

When  an  emergency  situation  occurred, 
he  took  the  necessary  actions  to 
render  immediate  aid  to  the  distressed 
aircraft  and  to  facilitate  search  au)d 
rescue  operations. 


Preouency 


U 


& 


3 


2 
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Effective  Job  Performance  of  Intercept 

Directors  as  Related  to  the  WD  Position  Frequency 


12.  Assumed  responsibility  to  perform  a 
task  ordinarily  done  by  someone 
else  and  was  successful  in  doing  so. 

He  acted  quickly  to  perform  a  task  1 

ordinarily  done  by  someone  else 

vhen  he  recognized  that  a  situation 

or  condition  woxild  become  further 

aggravated  without  immediate  action.  ___ 

Total  for  Effective  Job  16 

Performance  of  IHD 


Ineffective  Job  Performance  of  Senior 
Weapons  Directors  as  Related  to  the  WD 

Position  Frequency 


1.  Interfered  with  the  functions  of  the 
WD  by  excessive  and  inappropriate  use 
of  the  tactical  lines. 

He  would  call  to  obtain  Information  lU 

■vdilch  could  be  more  readily  obtained 

from  other  sources  (e.g. ,  SIDs  and 

DIDs);  to  inquire  about  Information 

without  allowing  sufficient  time  for 

the  information  to  be  obtained;  to 

Inquire  about  matters  unrelated  to 

the  immediate  situation;  and  to  ask 

questions  \dilch  could  be  attributed 

to  his  own  lack  of  knowledge. 

2.  Took  action  without  obtaining  all 
the  information  necessary  for  the 
accomplishment  of  a  task. 

He  acted  before  he  had  obtained  all  1 

necessary  information  from  personnel^ 
display  sources^  etc.,  to  accoi!q>llsh 
the  task. 
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Ineffective  Job  Performance  of  Senior 
Weapons  Directors  as  Related  to  the  VD 
Position 

3.  Took  action  vithout  Informing  the 
personnel  vho  would  be  concerned 
vlth  or  affected  by  the  action. 

He  failed  to  inform  relevant  person¬ 
nel  before  acting,  causing  the 
original  condition  or  problem  to 
become  further  aggravated  and  causing 
new  problems  to  be  created. 

!(..  Palled  to  formulate  and/or  execute 
an  effective  operational  plan  for 
the  Weapons  Section. 

He  did  not  formulate  a  workable  plan 
for  the  Weapons  Section  and  did  not 
provide  appropriate  Instructions  to 
the  personnel  concerned.  He  failed 
to  carry  out  the  operation  according 
to  prior  plans  and  caused  confusion 
and  disruption  to  the  operation. 

Total  for  Ineffective  Job 
Performance  of  SWDs 


Effective  Job  Performance  of  Senior 
Weapons  Directors  as  Related  to  the  WD 
Position _ 

5.  Displayed  alertness  by  recognizing 
a  condition  or  situation  \dilch  could 
become  dangerous  etnd  notifying  the 
appropriate  Weapons  Director  for 
action. 

U^n  recognition  of  a  condition  or 
situation  \dilch  could  become  dangerous, 
he  Informed  the  responsible  Weapons 
Director  of  the  matter  and  provided 
him  with  directions  and  suggestions 
for  corrective  action. 


Frequency 


3 


5 


23 


Frequency 
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Effective  Job  Performance  of  Senior 
Weapons  Directors  as  Belated  to  the  WD 

Position _ Frequency 


6.  Assisted  the  Weapons  Director  by 
assuming  responsibility  for  co¬ 
ordination  vlth  the  appropriate 
facilities,  agencies,  and  personnel 
concerned  vlth  a  particular  situation. 

When  a  situation  arose  in  which  the  3 

responsible  Weapons  Director  was 
occupied  by  other  important  and 
Immediate  tasks  concerned  with  the 
situation,  he  assisted  the  WD  by 
assuming  responsibility  for  co¬ 
ordinating  with  the  appropriate 
facilities,  agencies  and  personnel 
concerned. 

7.  Anticipated  an  event  which  would 
provide  difficulties  during  a  mission 
€md  took  effective  preparatory  or 
preventive  action. 

While  monitoring  an  operation,  he  3 

anticipated  certain  events  that  would 

provide  difficulties  for  the  personnel 

Involved  in  perforalng  the  task  emd 

took  the  required  action  to  prevent 

or  alleviate  these  difficulties  \dien 

they  occurred. 

8.  Formulated  and  executed  an  effective 
operational  plan  for  the  Weapons  Section. 

He  formulated  a  workable  plan  for  the  2 

conduct  of  the  air  battle  by  the  Weapons 
Section  and  provided  appropriate  in¬ 
structions  to  €^1  personnel  concerned. 

He  executed  the  operation  according 
to  plan  by  effective  scheduling,  by 
effective  monitoring  of  the  operation, 
smd  by  effective  coordination  with 
the  personnel  concerned. 


15  Jaauaiy  1963 


-21- 


m-956 


Effective  Job  Performance  of  Senior 
Weapons  Directors  as  Rel.ated  to  the  UD 
Position _ 

9.  Maintained  effective  discipline  on 
the  use  of  the  tactical  lines. 

He  helped  maintain  discipline  on 
the  use  of  the  tactical  lines  by 
establishing  "guidelines*'  on  the 
types  of  calls  to  be  made  and  the 
types  of  calls  to  be  excluded. 

Total  for  Effective  Job 
Performance  of  SWDs 


Frequenc: 


1 


12 


B.  Statistical  Data  concerning  the  Incidents  and  the  Respondent  Population 

An  analysis  vas  made  of  the  type  of  mission  In  \dilch  the  Incidents 
occurred.  Elghty>two  Incidents  occurred  during  simulated  missions 
and  76  occurred  during  live  exercises.  An  examination  of  the  recency 
of  the  occurrence  of  the  Incidents  vlth  respect  to  the  time  of  report 
vas  made.  The  results  are  shown  In  Table  II. 


TABLE  II 

Recency  of  Incident  Occurrence 
with  respect  to  Time  of  Report 


Time  Interval. 


Frequency  of 
Incidents 


S-i  month 
IS.  2  months 
'S3  months 
SI4.  months 

55  months 

56  months 
7  'to  12  months 
1  yetur  and  over 


The  respondent  population  vas  conqposed  of  'the  following:  Senior 
Weapons  Directors  (N-7)»  Weapons  Dlrec'tors  (Nb22),  Intercept  Directors 
(1^4o).  Table  III  presents  statistical  data  regarding  some  of  the 
characteristics  of  the  respondents  In  the  study. 
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TABLE  III 


Some  Characteristics  of  the  Respondent  Population 


Characteristic 

SWD  (N-7) 

WD  (N-22) 

Iim  (N-UO) 

Manual  experience 

M  ■  6.6  yrs. 

M  ■  h.2  yrs. 

M  »  1.8  yrs. 

SAGE  e3q>erlence 

M  ■  1.1  yrs. 

M  »  1.6  yrs. 

M  -  1.3  yrs. 

Total  Air  Defense  Ejqierlence 

M  -  7.7  yrs. 

M  ■  5*8  yrs. 

M  -  3.1  yrs. 

Time  since  conpletlon  of 

formed  manual  training^ 

M  »  4.1  yrs. 

M  ■  5*8  yrs. 

M  ■  3*2  yrs. 

Time  since  cooqiletlng 

SAGE  training**' 

M  ■  1.1  yrs. 

M  a  1.7  yrs. 

M  ■  1.3  yrs. 

Combat  Ratings'*** 

Training 

0 

1 

2 

Combat  Ready 

7 

21 

24 

Shilled 

— 

— — 

12. 

E:q>ert 

— 

— - 

2 

*  Means  were  calculated  excluding  those  who  did  not  undergo  any  formal  manual 
trednlng  (e.g..  Controller  School  at  Tyndall  AFB)  and  those  vho  did  not 
furnish  the  Information. 

**  The  original  query  vas  In  regard  to  formal  SAGE  training  (e.g.,  courses  at 
Keesler  AFB  or  Richards- Gebaur  AFB).  Since  a  leurge  proportion  of  respondents 
did  not  undergo  any  formal  training,  the  means  reflect  completion  of  training 
of  formal  and  on-slte  (OJT)  courses.  The  means  were  calculated  excluding 
those  \dxo  did  not  fUmlsh  the  Information. 

Combat  ratings  are  concerned  with  Intercept  Direction  capability.  Senior 
Weapons  Directors  and  Weapons  Directors  are  classified  In  either  confbat  ready 
or  training  status  since  they  normally  are  not  engaged  In  Intercept  direction. 
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C.  Incidents  obtained  from  the  Weapons  Director  Operational 
Procedures  and  Equlnment  Questionnaires 

A  total  of  64  Incidents  related  to  pro^blems  concerning  equipment, 
con^uter  program  and  standard  operating  procedures  were  obtained 
from  these  questionnaires. 

Oiese  Incidents  provided  a  listing  of  20  different  problems  and 
related  suggestions  for  Inqprovement  (Ref.  5)»  Six  of  the  problems 
had  been  corrected  or  were  In  the  process  of  being  corrected  by 
appropriate  equipment  or  program  changes.  Seven  problems  were  con¬ 
sidered  appropriate  for  the  recommendation  of  corrective  action.  Hie 
remaining  problems  and  related  suggestions  were  rejected  for  the 
following  reasons: 

1.  Implementation  of  the  suggested  change  was 
not  considered  feasible. 

2.  The  recommended  change  and  desired  results 
were  Incompatible. 

3.  The  problem  was  concerned  with  a  program  error 
peculiar  to  a  location. 

V.  DISCUSSION 

The  critical  Incident  technique  was  used  to  obtain  reports  of  the  following 
types:  l)  Incidents  concerning  the  performance  requirements  of  the  Weapons 
Director's  position,  2)  incidents  in  which  a  Senior  Weapons  Director's 
action  had  provided  an  aid  or  hindrance  to  the  effective  functioning  of  the 
Weapons  Director,  3)  Incidents  In  which  an  Intercept  Director's  action  had 
provided  an  aid  or  hindrance  to  the  effective  functioning  of  the  Weapons 
Director,  4)  Incidents  In  which  constraints  or  limitation  of  the  equipment, 
computer  program  or  standard  operating  procedmres  had  provided  a  hindrance 
to  the  WD  function.  The  behaviors  abstracted  from  the  Incidents  described 
In  items  1,  2,  and  3  above,  were  formed  Into  categories  and  listed  under 
the  appropriate  aspects  of  the  Weapons  Director's  function,  e.g. ,  WD  per¬ 
formance  requirements;  WD's  Interactions  with  SWDs  and  INDs.  The  Incidents 
concerned  with  the  equipment,  SCFs,  etc.,  provided  a  list  of  problems  6uid 
suggestions  for  change  associated  with  that  aspect  of  the  function.  An 
examination  of  the  content  of  the  Incidents  and  consideration  of  the  fre¬ 
quency  of  incidents  occurring  within  a  category  provided  a  means  for  identi¬ 
fying  and  describing  some  of  the  problem  areas  associated  with  the  position. 
The  frequencies  of  behaviors  within  a  category  provided  some  Indication  of 
the  relative  importance  of  the  Item.  It  should  be  understood  that  frequency 
does  not  solely  determine  \diether  a  given  category  of  behavior  Is  Important. 
However,  If  the  frequency  Is  large,  it  Is  difficult  to  assume  that  the  item 
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is  of  no  importemce.  For  exaa^ile,  if  a  large  proportion  of  personnel 
indicated  a  particular  behavior  as  being  Ineffective  or  a  particular  piece 
of  equipment  as  being  defective,  it  would  be  difficult  not  to  consider 
these  items  as  relevant  to  some  problems  associated  with  the  position. 

A  description  of  some  of  the  problem  areas  which  were  identified  are  pre¬ 
sented  below: 

a.  Area  assignment  for  INDs 

Assigning  INDS  to  interceptors  in  widely  separated  areas  causes 
a  large  number  of  difficulties  for  the  INDs  in  monitoring  and 
conducting  Intercepts.  When  the  Intercepts  progress  badly  or 
are  missed  as  a  result  of  difficulties  in  control,  the  VD  obtains 
a  large  amount  of  additional  work  in  contending  with  the  situation 
(e.g. ,  making  reassignments,  new  assignments,  etc.).  In  some  of 
the  reports  the  probability  of  assignments  being  made  in  widely 
separated  areas  was  enhanced  by  assigning  the  WD  to  air  bases  at 
opposite  ends  of  the  sector. 

b.  Commitment  of  Interceptors  to  targets 

In  some  of  the  cases,  the  WD  made  commitments  of  Interceptors  to 
targets  that  were  immediately  displayed  as  Iniposslble  according 
to  program  calculations.  Again,  the  WD  made  addltloned  work  for 
himself  in  making  new  commitments  and  "straightening  out"  the 
confusion  created  for  the  INDs.  EnQihasIs  should  be  placed  on  the 
use  of  appropriate  Information  displays  in  preventing  such  types 
of  conmltments  and  reducing  the  frequency  of  recommit  actions. 

c.  Distributing  assignments  and  maintaining  even  loads  on  INDs 

Ineffective  utilization  of  the  Intercept  Direction  capacity  and 
the  ensuing  related  problems  can  be  attributed  to  poor  distribution 
of  assignments  and  uneven  locds  on  INDs.  This  is  not  meant  to 
loply  that  all  INDs  shovild  get  the  same  number  of  aircraft  to  con¬ 
trol,  since  aircraft  control  capabilities  vary  among  different  INDs. 
However,  each  director  should  receive  a  load  ccmnnensurate  with  his 
capacity  so  maximum  utilization  of  the  weapons  team  capability  is 
achieved. 

d.  Formulation  of  adequate  "pre-mission  plans" 

In  the  incidents  described  by  the  category  "Lost  control  over  a 
situation  for  vhlch  he  was  responsible,"  a  major  causal  factor 
was  the  lack  of  definitive  and  specific  pre-mission  "work  plans” 
(within  the  context  of  the  major  battle  plan)  for  the  weapons  team. 
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The  lack  of  adequate  pre-mioslon  planning  causes  the  weapons  team 
to  function  in  a  disorganized  manner  eind  requires  the  WD  to  spend 
too  much  time  during  the  air  battle  dispensing  instructions  and 
information  to  the  INDs  which  could  have  been  disseminated  earlier. 
When  a  disproportionate  amount  of  time  is  spent  in  this  activity, 
the  WD  often  cannot  attend  to  and  deal  quickly  with  potentially 
dangerous  conditions  when  they  arise.  When  these  conditions  are 
not  effectively  contained,  loss  of  control  results.  Incidents  of 
effective  behavior  regarding  this  activity  were  reflected  in  the 
category  "Formulated  and  executed  an  effective  work  plan  for  his 
weapons  team." 

e .  Interfering  with  the  functions  of  the  WD  by  excessive  and  in» 
appropriate  use  of  tlie  tactical  lines 

A  large  number  of  cases  involving  the  SWD  pertain  to  this  area. 

In  many  of  the  incidents  the  SWD  did  not  serve  effectively  as  a 
"buffer"  in  reducing  the  number  of  calls.  Queries  initiated  by 
the  Battle  Staff  were  submitted  to  the  WD  by  the  SWD  without  con¬ 
sideration  of  the  activity  in  which  the  WD  was  engaiged.  The 
relevance  of  the  query  in  regard  to  the  present  situation  or  the 
immediate  task  was  frequently  Ignored.  Other  calls  generated  by 
the  SWD  are  revealing  of  the  fact  that  plans  for  the  weapons  teams 
in  conducting  the  air  battle  were  inadequate  or  had  not  been  pro¬ 
vided.  Interfering  calls  from  the  INDs  were  indicative  of  situations 
in  which  the  IND  was  unsure  of  himself  or  did  not  have  the  required 
knowledge  to  operate  his  position. 

A  plan  to  effect  and  maintain  telephone  discipline  may  help  in 
alleviating  a  large  number  of  problems  related  to  this  area. 

f .  Heav?/  overprinting  of  symbology  and  difficulties  in  read-throufdi 

The  need  for  a  X8  expansion  for  the  WD  console  is  indicated  by 
the  large  number  of  reports  related  to  the  symbology  overprinting 
problem.  Among  the  reports  collected  by  the  WD  Operational  Pro¬ 
cedures  and  Equipment  Questionnaire,  l6  incidents  were  related  to 
this  problem  and  all  recommendations  called  for  the  provision  of 
X8  as  a  solution. 

Heavy  overprinting  causes  many  problems  for  the  WD  in  reading 
tre«:k  symbologj-  and  results  in  considerable  delay  in  taking 
appropriate  console  actions.  One  of  the  effective  behaviors  of 
the  WD  was  described  by  the  category  "Assisted  another  Weapons 
Director  (WD)  by  effectively  monitoring  the  'gross'  air  picture." 

If  use  of  the  X8  display  produces  a  tendency  among  the  WDs  to 
confine  their  monitoring  activities  to  small  areas  of  the  scope. 
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the  benefits  which  result  from  the  behavior  described  above  will 
be  negated.  It  would  appear  that  discriminate  use  of  the  X8  dis¬ 
play  should  be  beneficial  In  obtaining  rapid  read-throu£^. 
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CRITICAL  INCIDENT  FORM  (WD-E) 

I.  Hiink  about  the  live  and  simulated  (STP)  missions  in  which  you  participated, 
or  in  which  you  had  a  chance  to  observe  a  Weapons  Director  working  during 
the  past  six  months.  Try  to  recall  a  situation  or  occasion  in  which  a  ^ 

Weapons  Director  did  something,  or  in  which  scmething  occurred  that  made 
you  feel  the  Weapons  Director  was  very  effective.  The  Weapons  Director 
need  not  have  performed  100%  effectively  in  all  phases  of  the  mission. 

What  is  important  is  that  the  behavior  you  observed  in  that  peurticular 
situation  was  superior  to  the  behavior  displayed  by  other  Weapons  Directors 
when  similar  situations  occurred.  The  Director  may  have  been  yourself  or 
another  Weapons  Director  you  were  observing.  In  either  case,  you  should 
be  quite  knowledgeable  of  the  circumstances  and  what  the  Weapons  Director 
did. 

On  the  next  page  describe  the  most  recent  situation  you  observed  in  which 
you  felt  a  Weapons  Director  was  very  effective. 
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CRITICAL  INCIDEMT  FORM  (WD-E) 

In  writing  your  report,  please  provide  as  much  detail  as  possible. 

A.  Exactly  ^diat  was  the  situation  or  the  surrounding  circumstances? 


B.  What  did  the  Weapons  Director  do? 


C.  Why  would  you  consider  this  behavior  as  being  very  effective? 


(After  con^jleting  the  description  be  sure  to  answer  the  questions  on  the  next 
page.) 
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Since  we  would  like  to  get  Information  from  different  Weapons  Directors 
with  different  eiqperlences,  we  would  like  the  following  Information: 

(a)  Approximately  how  long  ago  did  the  incident  occur?  _ 

(b)  In  >rtiat  type  of  mission  did  the  incident  occur?  SIM  LIVE 

(c)  What  was  the  experience  level  of  the  Weapons  Director? 

Training  statue  Combat  Ready _ 

(d)  How  long  has  it  been  since  you  completed  Weapons  Controller  training? 

Manual _  Where 

SAGE _  Where 

(e)  What  is  your  ci'rrent  status? 

Training _  Combat  Residy _ 

(f)  How  many  yeare  have  you  worked  in  SAGE? _  Manual _ 
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.CRITICAL  INCIDEMT  FORM  (WD-I) 

II.  Now  consider  the  opposite  type  of  behavior.  Try  to  recall  a  situation 
or  occasion  In  which  a  Weapons  Director  did  something  or  In  which  some¬ 
thing  occurred  that  made  you  feel  the  Weapons  Director  was  very  In¬ 
effective  during  the  course  of  the  mission.  The  Weapons  Director  need 
not  have  performed  lOOjt  Ineffectively  in  all  phases  of  the  mission.  What 
Is  Important  is  that  the  behavior  you  observed  in  that  particular  situ¬ 
ation  was  inferior  to  the  behavior  displayed  by  other  Weapons  Directors 
when  similar  situations  occurred.  The  Director  may  have  been  yourself  or 
another  Director  you  were  observing.  In  either  case  you  should  be  quite 
knowledgeable  of  the  circumstances  and  what  the  Weapons  Director  did. 

On  the  next  page  describe  the  most  recent  situation  you  observed  in  which 
you  felt  a  Weapons  Director  vas  ineffective. 
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CRITICAL  INCIDENT  FORM  (VfD-l) 

In  writing  your  report^  please  provide  as  much  detail  as  possible. 

A.  Exactly  ^diat  was  the  situation  or  the  surrounding  circumstances T 


What  did  the  Director  do? 


C.  How  do  you  think  the  situation  ml{^t  have  been  handled,  or  what  might 
the  Director  have  done  instead? 


(After  casqpletlng  the  description  be  sure  to  answer  the  questions  on  the  next 
page.) 
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Since  we  would  like  to  get  information  from  different  Intercept  Directors 
with  different  experiences,  we  would  like  the  following  infonnation: 

(a)  Approximately  how  long  ago  did  the  incident  occur?  _ 

(b)  In  what  type  of  mission  did  the  incident  occur?  SIM _  Live]^ _ 

(c)  What  was  the  experience  level  of  the  Weapons  Director? 

Training  status  Combat  Ready  ________________ 

(d)  How  long  has  it  been  since  you  completed  Weapons  Controller  training? 

Manual _  Where_ _ 

SAGE  Where _ 

(e)  What  is  your  current  status? 

Training _  Combat  Ready _ 

(f)  How  many  years  have  you  worked  in  SAGE? _ ^Manual _ 


15  January  1963 


-33- 


7M-956 


Appendix  B 


WEAPONS  DIRECTOR  OPERATIONAL  PROCEDURES 
.AND  EQUIPMENT  QUESTIONNAIRE 

The  effective  performance  of  a  Job  Is  affected  by  certain  conditions  ^Ich 
operate  within  the  SAGE  environment  other  than  the  people.  Kiere  have  prob¬ 
ably  been  situations  ^Ich  have  arisen  In  the  past  In  idilch  you  felt  your  per¬ 
formance  could  have  been  better  had  certain  conditions  been  Improved  or  modi¬ 
fied.  Specially,  we  are  concerned  with  those  conditions  brou^t  about  by  the 


following  factors; 

1. 

Console  equipment 

3- 

Communications  equipment 

2. 

Coniputer  program 

k. 

Standard  operating  procedures 

Try  to  recall  a  particular  situation  in  which  you  felt  that  one  of  the  factors 
mentioned  above  caused  a  condition  which  hindered  you  in  performing  your  Job, 

On  the  following  pages  describe  the  situations  In  detail  telling  what  happened, 
^diat  fsustors  were  Involved,  how  the  condltlonsCs)  mlf^t.  be  inqproved  so  that 
moire  efficient  perfomance  can  take  place. 
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WEAFONS  DIRECTOR  OFERATIONAL  PROCEDURES 
AM)  EQUIPMENT  QUESTIONNAIRE 

Please  write  only  one  incident  on  this  page. 

1.  Wliat  factor  was  involved? 


2.  What  happened,  or  what  condition  or  situation  occurred  that  hindered  you 
in  performing  your  Job?  (Describe  in  detail.) 


I 

I 

I 

3.  Hofw  mi^t  the  condition  be  iii5)roved  to  allow  you  to  perform  more 
efficiently? 
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Appendix  C 


A  Description  and  Guide  for  the  nlstratlon 
of  the  Critical  Incident  ftieatlonnalrea 


Introduction 

As  a  part  of  the  Weapons  Director  workload  project,  a  modification  of  the 
Critical  Incident  Technique*  is  being  en5)loyed  as  one  of  the  data  gathering 
methods.  Data  collected  by  this  method  should  help  to  uncover  problem  eureas 
of  the  Weapons  Director's  position  in  two  respects:  l)  in  terms  of  the  WD's 
function  and  his  Interactions  with  the  Senior  Weapons  Director  and  the  Inter¬ 
cept  Directors,  and  2)  in  terms  of  the  console  equipment,  communications 
equipment,  coiiq)uter  programs  and  standard  operating  procedures. 

The  critical  Incident  technique,  in  general  terms,  consists  of  the  collection 
of  reports  of  observed  behavior  in  specific  situations  which  are  considered 
extremely  effective  or  ineffective  in  the  performance  of  a  Job.  A  brief 
explanation  of  the  questionnaire  forms  utilized  In  this  project  ml^t  serve 
to  Indicate  the  adaptation  and  use  of  the  basic  technique  which  has  been  made 
for  the  present  study. 


The  Questionnaire  Forms 

Six  types  of  critical  incident  questionnaires  have  been  adapted  for  the  project. 
Three  of  the  questionnaires  are  designed  to  obtain  reports  from  Senior  Weapons 
Directors,  Weapons  Directors,  and  Intercept  Directors,  concerning  effective 
and/or  ineffective  behaviors  they  observed  of  WD  performance . 

Two  questionnaire  forms  are  utilized  to  obtain  data  concerning  the  WD  and  his 
interactions  with  the  SWD  and  the  IltDs.  On  one  of  the  forms,  the  Weapons 
Director  is  asked  to  report  a  specific  situation  in  which  an  SWD's  action  had 
either  aided  or  provided  a  detriment  to  his  effective  functioning.  On  the 
other  form  the  Weapons  Director  is  asked  the  same  question  in  rega^  to  an 
IND's  behavior. 


♦  Psychological  Bulletin,  Vol.  51,  No.  4,  July  1954,  p.  327-358. 
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The  Questionnaire  Forms  (Cont'd) 

The  sixth  form,  which  is  administered  to  Weapons  Directors,  is  used  to  obtain 
data  that  would  help  to  uncover  any  problems  associated  with  console'  equipment, 
communications  equipment,  computer  programs,  and  standard  operating  procedures. 
The  Weapons  Director  is  asked  to  report  on  a  specific  situation  in  \diich  a 
deleterious  condition  arose  as  a  result  of  shortcomings  attributable  to  any  of 
the  factors  mentioned  above. 

A  table  of  questionnaire  forms  to  be  used  and  their  purposes  will  be  presented 
in  Table  1  as  an  aid  for  the  proper  disposition  of  them. 

Table  1 


Form  # 

Form  Name 

Respondent 
(Person  to  whom 
form  is 
administered) 

Purpose 

m-kklj/OGO/OO 

WD  equipment  and  opera¬ 
tional  procedures 
•  questionnaire 

WD  only 

For  reports  of 
incidents  concern¬ 
ing  console,  com¬ 
munications  equip¬ 
ment,  programs, 

SOPs,  etc. 

AR-l+477/OOl/CX) 

Critical  Incident  Form 
(WD-IND-E) 

(WD-IND-I) 

WD 

Reports  of  incidents 
concerning  inter¬ 
actions  with  IND. 

AR- 4477/002/00 

Critical  Incident  Form 
(WD-SWD-E) 

(WD-SWD-I) 

WD 

Reports  of  incidents 
concerning  inter¬ 
actions  with  the  SWD. 

AH- 4477/003/00 

Critical  Incident  Form 
(WD-E) 

(WD-I) 

WD 

Reports  of  incidents 
concerning  Weapons 
Director  performance 

AR- 4477/004/00 

Critical  Incident  Form 
(IHD-WD-E) 

(IND-WD-I) 

IND 

Reports  of  incidents 
concerning  WD  per¬ 
formance  . 

AR- 4477/005/00 

Critical  Incident  Form 
(SWD-WD-E) 

(SWD-WD-I) 

SWD 

Reports  of  incidents 
concerning  WD  per- 
f ozmance . 
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Ouldellnee  for  the  Administration  of  QueBtlonnaireB 

The  guidelines  listed  below  are  provided  to  assist  the  reader  In  the  adalnlS” 
tratlon  of  the  questionnaires: 

1.  Brief  group  of  respondents  on  the  nature  of  the  project 
and  set  up  a  schedule  for  the  individual  adnlnlstratlon 
of  the  questionnaires. 

2.  Administer  q^uestionnalre  to  respondents  individually. 

This  procedure  has  been  suggested  as  a  result  of  a 
preliminary  administration  of  the  questionnaires  at 
LAADS.  Individual  interviews  enhanced  respondents' 
motivation  and  were  found  to  be  much  more  effective 
for  incident  production. 

3.  Assure  the  respondents  of  the  anonymity  of  the  data. 

Rsmes  are  not  to  be  used  in  any  portion  of  the  report. 

The  respondents  are  also  not  to  Include  their  names. 

4.  Check  to  insure  that  incidents  are  reported  according 
to  specifications.  A  ccnqplete  description  (e.g. ,  what 
occurred;  what  the  person  did;  why  the  respondent  felt 
the  behavior  was  effective  or  ineffective,  how  the 
situation  mi^t  have  been  handled  otherwise,  etc.)  is 
needed  for  a  usable  report.  Also,  be  sure  that  the 
8anq>le  control  data  section  following  each  incident 
description  is  filled  out. 

5.  It  is  desirable  to  obtain  at  least  two  reports  (one  of 
effective  and  one  of  ineffective  behavior)  on  each  of 
the  forms  from  each  of  the  respondents.  An  exception 
being  the  WD  equipment  and  operational  procedure  question¬ 
naire  where  reports  of  incidents  concerning  inadequacies 
of  console  equipment,  ccnmunlcatlon  equipment,  cooq^uter 
programs  or  SGPs  are  desired.  If  any  respondent  can 
provide  more  incidents,  he  should  be  encouraged  to  do  so 
and  shotild  be  supplied  with  additional  forms. 

Briefing  for  the  Respondents 

A  briefing  (see  Appendix  A  and  B)  should  be  given  to  the  respondents  for  the 
following  purposes : 

1.  To  esqplaln  the  nature  axid  pxupose  of  the  study. 

2.  To  give  instructions  for  filling  out  the  questionnaires. 
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3.  To  give  examples  of  well  written  and  poorly  written  incident 
reports  as  an  aid  to  the  respondents  for  identifying  and 
writing  out  critical  incidents.  By  directing  the  respondents' 
attention  to  specific  results  following  effective  or  ineffective 
behavior,  it  becomes  possible  for  them  to  isolate  and  describe 
the  relevant  behavior  or  behaviors  concerned.  This  should  help 
to  make  clear  the  "meaning"  of  critical  incidents  as  well  as  the 
requirements  for  writing  the  reports. 
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Appendix  C 


*Brleflng  for  WDs 

We  (the  System  Development  Corporation,  with  APASTO  and  ADC)  are  engaged  In  a 
stu^  to  uncover  certain  problem  areas  associated  with  the  Weapons  Director's 
function.  To  obtain  information  \diioh  mig^t  help  in  providing  a  focus  upon 
the  relevant  problem  areas  associated  with  the  WD  function,  we  are  asking  many 
Weapons  Directors,  like  yourselves,  Trom  different  direction  centers  in  differ¬ 
ent  areas  of  the  country  to  assist  us  by  responding  to  several  questionnaires. 
One  of  the  ways  in  which  we  can  obtain  Information  on  these  problem  areas  is 
to  study  specific  situations  in  which  either  very  effective  or  ineffective 
behavior  was  demonstrated  by  a  WD  in  performing  his  Job.  On  one  of  the  forms 
we  would  like  you  to  describe  at  least  two  incidents  in  which  you  obseirved  an 
effective  and/or  ineffective  behavior  concerning  WD  performance.  Another 
aspect  of  the  problem  is  to  study  situations  which  involve  interactions  be¬ 
tween  Weapons  Directors  and  the  SWD  and  IMDs.  Two  forms  are  provided  for  these 
purposes.  On  one  of  these  forms  we  would  like  you  to  describe  two  situations 
or  incidents  in  which  a  SWD  had  either  aided  or  provided  a  detriment  to  yoxir 
functioning.  On  the  other  form  we  would  like  at  least  two  reports  of  incidents 
involving  the  IHD.  On  the  fourth  and  final  form  we  would  like  you  to  describe 
several  situations  in  which  inadequacies  of  either  console  equipment,  communi¬ 
cations  equipment,  computer  programs,  or  SOPs  caused  a  condition  which  hajiq>ered 
you  in  the  performance  of  your  Job. 

In  writing  your  reports  do  not  include  any  names  in  your  descriptions.  Please., 
do  not  include  your  name  on  the  report.  Ihe  reports  we  get  from  you  will  be 
used  only  for  the  purposes  of  this  research.  You  may  have  a  few  questions 
you  would  like  to  ask  at  this  polht.  Before  we  get  to  answering  them, perhaps 
a  few  exaiiq)les  of  critical  Incident  reports  may  help  to  answer  some  of  your 
questions. 


*  It  is  not  Intended  that  this  briefing  be  delivered  verbatim,  but  is  provided 
as  cm  aid  to  the  administrator  of  the  questionnaires  to  Insure  that  all  perti¬ 
nent  points  are  covered. 
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1.  A  veil  written  report  of  Ineffective  behavior 

During  a  misBion  with  a  heavy  load  of  traffic  entering  the  sector, 
the  WD  became  engrossed  In  the  replacement  of  aircraft  as  they  were 
becoming  low  on  fuel.  A  situation  developed  when  he  assigned  a 
flight  of  Interceptors  in  the  far  northern  part  of  the  sector  and 
another  flight  in  the  far  southern  part  of  the  sector  to  the  same 
IND  for  control  against  targets  in  those  areas.  This  action  nec- 
cessitated  the  IND  to  continually  switch  his  display  to  different 
parts  of  the  sector  and  restricted  his  use  of  the  X8  expanded  mode. 

It  also  made  it  necessaiy  for  the  IND  to  use  two  radio  sites  instead 
of  one.  Iftider  these  conditions  the  IND  missed  a  number  of  intercepts. 

I  think  the  WD  should  have  taken  a  reassignment  action  or  should  have 
effected  a  transfer  of  fighters  so  that  the  IND  could  control  filters 
that  were  in  one  portion  of  the  sector.  The  WD  should  try  when  possible 
to  Insure  that  all  missions  assigned  to  an  IND  are  confined  to  one  part 
of  the  sector. 

Comment ;  This  report  is  well  written  since  you  clearly  understand 
what  the  situation  was,  what  happened,  what  the  WD  did, 
and  what  the  person  thought  the  WD  should  have  done 
Instead. 


2.  A  well  written  report  of  effective  behavior 

The  Weapons  Director  of  Team  2  was  working  the  northern  part  of  the 
sector  >diere  the  bulk  of  the  traffic  was  coming  in.  The  INDs  on  his 
team  were  nearly  fully  loaded  with  interceptors  under  control.  The 
SWD  had  gone  into  the  automatic  assignment  mode  and  WD-2  kept  receiving 
more  Fakers.  At  this  point  WD-3  called  WD-2  and  offered  him  inter¬ 
ceptors  which  he  (WD-3)  had  positioned  earlier.  This  lessened  the 
load  on  WD-2'b  position  and  averted  an  overload  situation  from  develop¬ 
ing.  By  effective  monitoring  of  the  air  sitiistlon  and  by  recognizing 
a  load  condition  developing  at  WD-2'8  position,  WD-3'8  quick  action 
averted  an  extreme  overload  from  developing  which  helped  significantly 
towards  the  success  of  the  mission. 


Comment:  Again,  this  report  tells  us  \rtiat  happened,  what  the  WD  did 
and  the  behavior  was  particularly  effective. 
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3.  A  poorly  written  report  of  effective  behavior 

The  WD  during  the  last  mission  seemed  particularly  effective. 

Throughout  the  mission  he  demonstrated  a  good  knowledge  of  operational 
procedures.  He  maintained  cool  composure  under  difficult  circumstances 
and  displayed  initiative  in  many  instances  throughout  the  mission. 

Comment ;  This  report  is  poorly  written,  primarily  because  it  lacks 
anything  specific.  The  terms  "demonstrated  good  knowledge 
of  operational  procedures,"  "displayed  initiative,"  tells 
us  very  little  of  eMictly  vrtiat  occur;f.ed  or  vdiat  the  Director 
did  in  being  effective. 


Are  there  any  questions  in  regard  to  these  questionnaires? 

(Directions  may  now  ba  given  as  to  when  each  individual  will  be  interviewed 
for  the  completion  of  the  questionnaires,  etc.). 
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♦Briefing  for  SWPs  and  INDs 

We  (the  System  Development  Corporation,  with  APASTO  euad  ADC)  are  engaged  in  a 
study  to  uncover  certain  problem  areas  associated  with  the  Weapons  Director's 
function.  By  studying  specific  situations  in  which  either  very  effective  or 
ineffective  behavior  was  demonstrated  by  a  Weapons  Director  in  performing  the 
Job,  we  can  get  information  which  may  help  in  focusing  on  the  problem  areas 
associated  with  this  function.  During  our  study  we  are  asking  many  persons, 
like  yourselves,  who  interact  to  a  great  extent  with  Weapons  Directors,  to 
describe  several  recent  situations  in  \diich  either  type  of  behavior  occurred. 
Since  you  are  involved  in  some  way  with  the  Weapons  Director's  function,  we 
feel  you  are  a  very  appropriate  source  for  obtaining  this  information. 

In  your  job  as  an  IND  or  SWD  you  have  probably  observed  or  experienced  a  few 
situations  about  which  you  can  report. 

In  writing  your  reports  do  not  include  any  names  in  your  descriptions.  Please 
do  not  include  your  name  on  the  report.  The  reports  we  get  from  you  will  be 
used  only  for  the  purposes  of  this  research.  You  may  have  a  few  questions 
you  would  like  to  ask  at  this  point.  Before  we  get  to  answering  them,  perhaps 
a  few  examples  of  critical  incident  reports  may  help  to  answer  some  of  your 
questions. 

1.  A  well  written  report  of  ineffective  behavior 

During  a  mission  with  a  heavy  load  of  traffic  entering  the  sector, 
the  WD  became  engrossed  in  the  replacement  of  aircraft  as  they  were 
becoming  low  on  fu.el.  A  situation  developed  when  he  assigned  a 
flight  of  interceptors  in  ilie  far  northern .part  of  the  sector  and 
another  flight  in  the  far  southern  part  of  the  sector  to  the  same 
IND  for  control  against  targets  in  those  areas.  This  action  nec- 
cessitated  the  IND  to  continually  switch  his  display  to  different 
parts  of  the  sector  and  restricted  his  use  of  the  X8  expanded  mode. 

It  also  made  it  necessary  for  the  IND  to  use  two  radio  sites  Instead 
of  one.  Under  these  conditions  the  IND  missed  a  nmber  of  Intercepts. 
I  think  the  WD  should  have  taken  a  reassignment  action  or  should  have 
effected  a  transfer  of  fighters  so  that  the  IND  could  control  fighters 


*  It  is  not  intended  that  this  briefing  be  delivered  verbatim,  but  is  provided 
as  an  aid  to  the  administrator  of  the  questionnaires  to  insure  that  all  perti¬ 
nent  points  are  covered. 
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that  were  in  one  portion  of  the  sector.  The  WD  should  try  when 
possible  to  insure  that  all  missions  assigned  to  an  IND  are  confined 
to  one  part  of  the  sector. 

Comment:  This  report  is  well  written  since  you  cle8n:'ly  understand 
what  the  situation  was,  what  happened,  what  the  WD  did, 
and  what  the  person  thou^t  the  WD  should  have  done 
instead. 

2.  A  well  written  report  of  effective  behavior 

The  Weapons  Director  of  Team  2  was  working  the  northern  part  of  the 
sector  where  the  bulk  of  the  traffic  was  coming  in.  The  INDs  on 
his  team  were  nearly  fully  loaded  with  interceptors  under  control. 

The  SWD  had  gone  into  the  automatic  assignment  mode  and  VD-2  kept 
receiving  more  Fakers.  At  this  point  WD-3  called  WD-2  and  offered 
him  interceptors  which  he  (WD-3)  had  positioned  earlier.  This 
lessened  the  load  on  WD>2's  position  and  averted  an  overload  situ¬ 
ation  from  developing.  By  effective  monitoring  of  the  air  situation 
and  by  recognizing  a  load  condition  developing  at  WD-2' a  position, 

WD-3' 8  quich  action  averted  an  extreme  overload  from  developing  which 
helped  significantly  towards  the  success  of  the  mission. 

Comment ;  Again,  this  report  tells  us  what  happened,  what  the  WD 
did,  and  why  the  behavior  was  particularly  effective. 

3.  A  poorly  written  report  of  effective  behavior 

The  WD  during  the  last  mission  seemed  particularly  effective.  Through¬ 
out  the  mission  he  demonstrated  a  good  knowledge  of  operational  pro¬ 
cedures.  He  maintained  cool  cco^osure  under  difficult  circumstamces 
and  displayed  initiative  in  many  Instances  throu^out  the  mission. 

Crnnmen-fc !  ihis  report  Is  poorly  written,  primarily  because  it  lawks 
anything  specific.  The  terms  "demonstrated  good  knowledge 
of  operational  procedures,"  "displayed  initiative,"  tells 
us  very  little  of  exawtly  what  occurred  or  what  the 
Director  did  in  being  effective. 

Are  there  amy  questions  in  regard  to  these  questionnaires? 

(Directions  may  now  be  given  as  to  >dien  each  individual  will  be  interviewed 
for  the  completion  of  the  questionnaires,  etc.). 
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Examines  three  aspects  of  the 
Weapons  Director  (WD):  l)  the 
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interactions  with  the  Senior  Weapons 
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